UNCLASSIFIED 


AD  NUMBER 


ADB069698 


NEW  LIMITATION  CHANGE 
TO 

Approved  for  public  release,  distribution 
unlimited 


FROM 

Distribution  authorized  to  U.S.  Gov't, 
agencies  only;  Test  and  Evaluation;  June 
1982.  Other  requests  shall  be  referred  to 
Director,  US  Army  Materiel  Systems 
Analysis  Activity,  Aberdeen  Proving 
Ground,  MD  21005. 


AUTHORITY 


USAMSAA  ltr,  dtd  20  Jul  1983 


THIS  PAGE  IS  UNCLASSIFIED 


• FfLF  COPY  ^ 69  6 98 


TECHNICAL  RF PORT  NO-  357 


AN  IMPROVED  METHODOLOGY  FOR  RELIABILITY 
GROWTH  PROJECTIONS 


LARRY  H.  CROW 


JUNE  1982 


DT1C 

ELECTE 
DEC  1 6 1982 


Distribution  limited  to  US  Government  agencies  only;  *Mu  i u ijcji  L"wviers* 
test  and  evaluation  -t-ir — iTr1  paaditfA*  m M,‘  jMl— 1 1 j 
June  1982.  Other  requests  for  this  document  must  be  referred  to  the 
Director,  US  Army  Materiel  Systems  Analysis  Activity,  Aberdeen  Proving 
Ground,  Maryland  21005. 


U.  S.  ARMY  MATERIEL  SYSTEMS  ANALYSIS  ACTIVITY 

ABERDEEN  PROVING  GROUND,  MARYLAND 


1 


DISPOSITION 


Destroy  this  report  when  no  lender  needed.  Do  not  return  it  to 
the  originstor. 


DISCLAIMER 

Tiie  findings  in  this  report  are  not  to  be  construed  as  an 
official  Department  of  the  Army  position  unless  sc  specified 
by  other  official  documentation. 


WARNING 

Information  and  data  contained  in  tnis  document  are  based  on 
the  input  available  at  the  time  of  preparation.  The  results  may  be 
subject  to  change  and  should  not  be  construed  as  representing  the 
DARCOM  position  unless  so  specified. 


TRADE  NAMES 

The  use  of  tr<*dn  names  in  this  report  does  not  constitute  an 
official  endor aemerit  or  approval  c f the  use  of  such  commercial 
hardware  or  software.  The  report  may  not  be  cited  for  purposes 
of  advertisement. 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  C*hon  Data  Enfarad; 


REPORT  DOCUMENTATION  PAGE 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


1.  REPORT  NUMBER 


Technical  Report  No.  357 


1.  GOVT  ACCESSION  NO. 

A.i)  ■ jj£l 


l.  RECIPIENT'S  CATALOG  NUMBER 


4.  TITLE  (end  Subtitle) 

An  Improveo  Methodology  for  Reliability 
Growth  Projections 


5.  TYPE  OF  REPORT  4 PERlOO  COVERED 


6.  PERFORMING  ORG.  REPORT  NUMBER 


7.  AuTHORfa) 

Larry  H.  Crow 


S.  CONTRACT  OR  GRANT  NUMBER! a,) 


9.  PERFORMING  ORGANIZATION  NAME  AND  AOORESS 


10.  PROGRAM  ELEMENT,  PROJECT.  TASK 
AREA  * WORK  UNIT  NUMBERS 


US  Army  Materiel  Systems  Analysis  Activity 
Aberdeen  Proving  Ground.  MD  21005 


DA  Project  No. 

LR7&5?QfaM54! 


II.  CONTROLLING  OFFICE  NAME  ANO  AOORESS 


US  Army  Materiel  Development  & Readiness  Command 
5001  Eisenhower  Avenue 

Alexandria.  VA  22333 


12.  REPORT  DATE 

June  1982 


13.  NUMBER  OF  PAGES 


14.  MONITORING  AGENCY  NAME  4 AOOHESS^/f  dlttm rmU  from  ControlUnj  Otficm) 


IS.  SECURITY  CLASS,  (ol  thlm  report) 

UNCLASSIFIED 


15a.  OECLASSI  FI  CATION/ DOWNGRADING 
SCHEDULE 


IS.  DISTRIBUTION  STATEMENT  fo<  Me  Report) 


Distribution  is  limited  to  US  Government  agencies  only;  i»npm-l- cirim' j Uln 
test  and  evaluation  of  mmmmmei al  ■ynadugtaww m 444aay  hai' rimin',  June  1982. 
Other  requests  for  this  document  must  be  referred  to  the  Director,  US  Army 
Materiel  Systems  Analysis  Activity,  Aberdeen  Proving  Ground,  MD  21005. 


17.  DISTRIBUTION  STATEMENT  < ’at  rha  attract  antarad  in  Block  20.  It  difformt  from  Raport; 


18.  SUPPLEMENTARY  NOTES 


18.  KEY  WORDS  ( Continue  on  f retoe  tide  II  neceeeery  and  Identify  by  block  number) 

Reliability  Management 
Growth  Tracking 
Fix  Effectiveness 
Reliability  Projection 


20*  ABSTRACT  rCoBfJtau*  «e»  aMv  ?f  rracttaaiy  mtd  Id  mu  tty  by  block  number) 

At  the  end  of  a development  test  phase  delayed  fixes  are  often  incorporate^ 
into  the  system  and  it  is  usually  desirable  to  make  a projection  of  the  impac 
of  these  fixes  on  the  system  reliability  at  the  beginning  of  the  next  phase  ofj 
devel opment.  ilhe  most  common  procedure  in  practice  for  making  reliability 
projections  utilizes  engineering  assessments  of  the  effectiveness  of  the  delated 
fixes  for  each  observed  problem  failure  mode.  These  effectiveness  factors 
are  then  used  with  data  generated  during  the  test  phase  to  obtain  a projected 


DO  , 


FOR**  U73 

JAM  73 


EDITION  OF  » NOV  S5  IS  OBSOLETE 


■ UNCI  ASSIF-lf-Q. 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (BW*  Drntm  Entered, ) 


UNCLASSIFIED 

StCuBITY  CLASH  FI  CATION  OF  THIS  P»QgfWH«n  Ojl«  Sntmnd) 

2(3"!  Abstract  (CONTINUED)  " 

v estimate  for  the  updated  configuration  by  adjusting  the  number  of  failures 
observed  during  the  test  phase.  Me  call  this  method  the  "adjustment  procedure 
(AP). 

In  this  report,  we  study  the  accuracy  associated  with  the  AP  for  reliabilit 
projections.  Results  of  this  study  show  that  there  are  two  sources  of  error 
associated  with  the  AP.  The  consequence  of  these  errors  is  that  the  AP  Is 
overly  optimistic,  generating  projected  values  which,  on  average,  over-estimat 
the  system  reliability.  In  particular,  it  was  found  that  even  when  the  effec- 
tiveness factors  are  known  exactly,  the  AP  will  still  overestimate  the  system 
reliability. 

A procedure  is  developed  in  this  report  for  estimating  the  error  term  of 
the  AP^"  Utilizing  this  procedure  a practical  projection  methodology  is  con- 
structed which  gives  improved  accuracy  for  makinq  reliability  projections. 

| Numerical  examples  illustrating  the  application  of  this  metLodol ogy  are  given. 


UNCLASSIFIED 


£ SECURITY  CLASSIFICATION  OF  THIS  PAGEr“*’ 


tmrmd) 


CONTENTS 


PAGE 


1.  ACKNOWLEDGMENT  A 

2.  INTRODUCTION  5 

3.  COMMENTS  ON  THE  ADJUSTMENT  PROCEDURE  6 

4.  A SYSTEM  RELIABILITY  PROJECTION  MODEL  10 

5.  CONCLUSIONS 16 

REFERENCES 17 

DISTRIBUTION  LIST 19 


Kor  __  ] 

r 


NT  IS  iVvii  L 
DTI  ' VAf. 
l)a. ‘Vi 

i ; . ' • 1 oil 


i' 


!■ 


. — i 


I 


!■ 


ACKNOWLEDGMENT 

The  author  wishes  to  thank  Mr.  Will  lam  J.  Broemm  for  the  computer 
programming  which  generated  the  data  for  the  numerical  examples. 


\U  IMPROVED  METHODOLOGY  FOR  RELIABILITY  GROWTH  PROJECTIONS 


I.  INTRODUCTION 

The  ne?d  for  highly  reliable  military  systems  is  obvious  and  this 
need  is  generally  reflected  in  high  reliability  requirements  to  be  attained 
during  development.  The  early  prototypes  for  complex,  military  systems  will 
invariably  have  significant  reliability  and  performance  deficiencies  ar,d, 
consequently,  these  systems  are  subjected  to  a development  testing  program 
to  find  problems  and  take  corrective  action.  The  improvement  in  reliability 
and  performance  which  occurs  will  depend  on  the  number  and  the  effectiveness 
of  the  fixes  that  are  incorporated  into  the  system. 

Experience  has  shown  that  programs  which  rely  simply  on  a demon- 
stration test  by  itself  to  determine  compliance  with  the  reliability  require- 
ments generally  do  not  achieve  the  reliability  objectives  with  the  allocated 
resources.  It  has  become  increasingly  clear  that  management  and  engineering 
attention  to  reliability  throughout  the  program  is  necessary  if  the  high 

requirements  for  complex  systems  are  to  be  met.  Reliability  growth  manage- 

■ ment  is  defined  in  US  Department  of  Defense  Handbook  189  as  "the  systematic 

' planning  for  reliability  achievement  as  a function  of  time  and  other 

j resources,  and  controlling  the  on-going  rate  of  achievement  by  reallocation 

! of  resources  based  on  comparisons  between  planned  and  assessed  reliability 

I values." 

I 

j Major  management  decisions  regarding  the  reliability  effort  are 

made  based  on  comparisons  of  the  assessed  and  target  values.  If  the  assess- 
i ments  are  in  agreement  with  the  target  values,  the  reliability  progrim 

j typically  will  remain  unchanged.  However,  if  the  assessed  values  are  well 

helow  the  target  values,  then  major  changes  in  the  program  may  be  necessary. 

If  the  assessed  values  do  not  accurately  reflect  the  system  reliability 
status,  then  clearly,  incorrect  management  decisions  can  be  made;  the  sys- 
tem may  be  accepted  and  fielded  with  lower  reliability  than  desired  or 
unnecessary  and  costly  changes  and  delays  in  the  program  may  occur. 

j For  most  development  programs,  reliability  assessment  methodol- 

| ogies  must  account,  for  a dynamic  environment  due  to  modifications  being 

I ' incorporated  into  the  system.  For  example,  in  the  presence  of  reliability 

j growth,  the  data  from  the  earlier  part  of  the  test  phase  would  not  be  repre- 

I sentative  of  the  current  configuration.  On  the  other  hand,  the  most  recent 

test  data,  which  would  best  represent  the  cur  ent  configuration,  may  be 
limited,  and  not  in  itself  be  sufficient  for  a valid  reliability  estimate. 
Because  of  this  situation,  various  growth  models  or  techniques  a'e  often 
employed  for  assessing  system  reliability  during  a test  phase. 

According  to  the  format  in  US  Department  of  Defense  Handbook  189, 
two  reliability  estimates  would  typically  be  made  at  the  end  of  a test 
phase.  The  demonstrated  reliability  value  is  an  estimate  of  the  system 
reliability  for  its  configuration  at  the  end  of  the  test  phase.  The  demon- 
strated value  is  based  on  data  generated  during  the  test  phase.  Also,  at 
the  end  of  the  test  phase  delayed  fixes  are  often  incorporated  into  the 
system  and  it  is  usually  desirable  to  make  a projection  of  the  impact  of 
these  fixes  on  the  system  reliability  at  the  beginning  of  the  next  phase 
i'  of  development. 
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Most  of  the  reliability  qrowj,  literature  has  been  concerned  with 
procedures  and  models  most  approprl  ■ ' *•  a demonstrated  reliability  value 
and  very  little  attention  has  been  p'.  to  techniques  for  reliability  projec- 
tions based  on  delayed  fixes.  The  >..,<» t common  procedure,  in  practice,  for 
making  reliability  projections  when  fixes  are  delayed  until  the  end  of  the 
test  phase,  utilizes  engineering  assessments  of  the  effectiveness  of  the 
delayed  fixes  for  each  observed  problem  failure  mode.  The  effectiveness 
factors  are  then  used  with  data  generated  during  the  test  phase  to  obtain  a 
projected  estimate  for  the  updated  configuration  by  adjusting  the  number  of 
failures  observed  during  the  test  phase.  Vo  call  this  method  the  "adjust- 
ment procedure."  Seo  US  Department  of  Defense  Handbook  189  for  a discussion 
of  reliability  growth  projection  procedures. 

In  this  report,  we  study  the  accuracy  associated  with  the  adjust- 
ment procedure  and  propose  an  improved  projection  model.  Specifically, 

’n  Section  2 we  define  a rigorous  structure  and  framework,  consistent  with 
real  life,  for  investigating  the  projection  procedures.  Based  on  this 
structure,  we  show  that  there  are  two  sources  of  statistical  bias  associated 
with  the  adjustment  procedure  which  generally  cause  one  to  overestimate  the 
system  reliability.  In  particular,  it  is  shown  that  even  when  the  effective- 
ness factors  are  known  exactly,  the  adjustment  procedure  is  still  a biased 
estimate.  In  Section  3 we  develop  an  improved  projection  model  which 
removes  one  important  source  of  error  inherent  in  the  adjustment  procedure. 

2.  COMMENTS  ON  THE  ADJUSTMENT  PROCEDURE 

Refore  we  describe  the  adjustment  procedure,  we  will  first  give 
some  background  and  establish  some  needed  notation. 

Suppose  a system  is  subjected  to  development  testing  for  a period 
of  time  T.  The  system  can  be  considered  as  consisting  of  two  types  of 
failure  modes.  Type  A modes  are  all  failure  modes  such  that  when  seen 
during  test,  no  corrective  action  will  be  taken.  This  accounts  for  all 
modes  for  which  It  is  not  cost-effective  to  attempt  to  increase  the  relia- 
bility by  a design  change.  Type  B modes  are  all  modes  such  that  if  seen, 
a design  change,  or  fix,  will  be  attempted. 

It  is  assumed  that  all  Type  B modes  are  in  series  and  fail  inde- 
pendently according  to  the  exponential  distribution.  We  also  assume  that 
the  occurrence  of  Type  A modes  follow  the  exponential  distribution  with 
failure  rate  a^.  In  this  paper,  we  will  assume  that  fixes  for  Type  B 
modes  found  during  test  will  be  incorporated  as  delayed  fixes  at  the  end 
of  the  test  phase. This  implies  that  the  system  reliability  is  constant 
throughout  the  test  phase  and  will,  then,  jump  to  a higher  value  after  the 
delayed  fixes  have  been  implemented. 

Lee  K denote  the  number  of  Type  B modes  in  the  system  and  let 

he  the  failure  rate  for  the  1-th  Type  B mode,  i = 1 K.  Then, 

at  time  0,  tne  system  failure  rate  r(0)  is 

r(G)  = \\  + kg,  (1 ) 


K 

where  Ag  = l X-j.  (In  practice,  of  course,  K will  generally  not  be 
i=l  known  before  or  after  the  testing.) 
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During  the  test  time  (0,  T)  a random  number  M <_  K of  distinct. 
Type  B modes  will  be  observed.  We  further  denote  by  dj  the  effectiveness 
factor  (EF)  for  the  i-th  Type  B mode,  f=1,...,K.  The  factor  di  Is  the 
percent  decrease  In  x,-  after  a corrective  action  has  been  made  for  the 
1-th  Type  B mode. 

Let  1 1 , i 2 . - • • . i M be  random  variables  which  denote  the  indices 
of  the  Type  R modes  observed  during  the  test  time  (0,T).  As  a result  of 
the  M corrective  actions  taken  on  the  M Type  B modes  observed,  the  system 
failure  rate  at  time  T is  reduced  from  r(0)  to 

M M 

r(T)  = xA  + v (i  - d ■) . )x  i . + (xB  - l Xi.l  (2) 

j=1  J J j = i j 


= Xa  + XB  - l d i X i . 

jxi  i j 


The  term 


1 ( 1 • d-j  . )X<  (4) 

jxl  J J 

is  the  failure  rate  for  the  M modes  after  the  corrective  actions.  The  term 

M 

(XB  - l Xi  .)  (5) 

j-1  J 

is  the  remaining  failure  rate  for  all  unseen  Type  B modes. 

The  adjustment  procedure  is  a method  which  has  been  used  as  an 
estimate  of  r(T).  We  will  show  that  this  procedure  is  not  valid,  and  that 
it  will  usually  yield  erroneous,  totally  misleading  results.  In  particular, 
we  show  that  the  adjustment  procedure  will  generally  overestimate  the  true 
current  system  reliability. 

Let  Na,  Nb  be  the  total  number  of  Type  A and  Type  B failures 
observed  and  let  N = NA  + Ng.  For  the  M <«_  NB  distinct  Type  B modes 
observed  during  test  we  let  N ,•  denote  the  number  of  observed  failures  for 

j 

B mode  ii.  The  adjustment  procedure  consists  of  reducing  the  Mi  for  the 
J j 

observed  Type  B modes  to  reflect  a decrease  in  failure  rate  resulting  from 

the  correction  action.  Let  d..-  denote  the  assumed  EF.  Then,  the  adjustment 

1 H 

o 

procedure  modifies  H by 

> * M 

■'  = Nft  + l (Hj  . )tJ,-  . (6) 

j = l J -1 

and  estimates  the  system  failure  rate  at  time  I by 


r.  -1 


(7) 


r*(T)=  N*/T 

* na/t 


M 


(1_dV  Nlj/T‘ 


(8) 


Observe  that  if  d!  = 1 (i.e.,  the  fixes  are  all  assumed  to  be 

j 

100  percent  effective),  then 

r*(T)  = Na/T  (9) 

which  simply  estimates  the  failure  rate  for  the  Type  A modes.  This  will 
certainly  underestimate  the  system  failure  rate  at  time  T unless  all  prob- 
lem failure  modes  (Type  B)  In  the  system  have  been  found  and  completely 
removed.  This  rarely,  if  ever,  happens  for  complex  systems  during  develop- 
ment. 


To  quantify  the  bias  for  the  adjustment  procedure  we  next  consider 
the  expected  value  of  r(T)  - r*(T).  Observe  that  r(T),  the  true  system 
failure  rate  at  time  T,  is  a random  variable  depending  on  the  Type  B modes 
seen  during  the  test  and  their  EFs. 


Let  1 -f  ( t ) denote  the  indicator  function  deffnsd  by 


lift)  = 


1 

0 


if  1-th  Type  B mode  occurs  during  (0,t) 
otherwise  (10) 


Then,  we  may  write  r(T)  as 


r(T)  « xA 


LI  - d-j I -j (T) J Xi  . 


(11) 


Also,  note  that  r*(T)  may  be  written  as 


Therefore, 


r (T!  = Na/T  + l (1  - dt)  N j/T. 
1=1 


E(r  (T) ) = E(Na/T  + V (1-dj)  Nf/T) 


= + 1 (1  - d1>  * 
< 1 = 1 


1 


and 


(12) 


(13) 

(14) 
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K 

t(r(T) ) = t(XA  + 1 [I  - di  1|(T)3  M)  (lb) 

1*1 

X -X  ■ T 

• *A  + * £1  - di(l  - e 1 )3  V (16) 

i=l 

This  yiel ds 

E(r(T)-r*(T))  * 

* * K -X  • T 

i.  (d,-  d1)x1  + i diXie  . (17) 

i=l  i=l 

Equation  (17)  gives  the  average  difference  in  the  adjustment  pro- 
cedure estimate  and  the  actual  system  failure  rate.  The  first  term  in  (17) 
is,  of  course,  the  contribution  to  the  bias  resulting  from  the  difference 
between  the  estimated  EFs  and  the  true  EEs.  The  second  term  is  the  contri- 
bution to  the  bias  resulting  from  the  randomness  associated  with  the  occur- 
rence of  Type  B problem  modes  during  the  test  period  (0,T). 

fiow,  it  is  very  important  to  note  that  E(r*(T) ) given  by  Equation 
(14)  does  not  depend  on  T.  That  is,  the  expected  value  of  the  adjustment 
procedure  estimate  is  a constant  for  all  T.  However,  the  expected  value 
of  the  actual  system  failure  rate  given  by  Equation  (16)  decreases  as  the 

test  time  T increases.  As  a matter  of  fact,  E(r(T))  approaches  t(r*(T)) 

as  T + °°  for  d*  = d^ . The  adjustment  procedure  estimate  r*(T)  when  d*  = d^ 

is  actual ly  an  estimate  of  the  limiting  system  failure  rate  when  all  Type  B 
modes  have  been  observed  and  a fix  incorporated.  This  implies  that  for 

d,  = d^,  r (T)  is  an  estimate  of  the  lower  bound  E(r  (T))  on  system  failure 
rate  and  is  not  a valid  estimation  procedure  fon  r(T),  the  current  system 
failure  rate. 

Assume  that  d*  = d^  and  note  that 

K. 

E(r  (T))  = X^  + Xg  - 2.  djXj.  (IB) 

i=l 

The  following  remarks  are  to  further  clarify  the  significance  of  the 
bias  associated  with  the  adjustment  procedure.  The  initial  failure  rate 
for  the  i-th  Type  B mode  is  Xj.  If  the  i-th  mode  is  observed,  the  value 
that  the  failure  rate  will  be  reduced  to  is  f 1 -d  -j ) X ^ after  the  fix;  that 

is,  Xj  is  reduced  by  the  amount  X -j . The  probability  that  the  i-th  mode 

-Xi  T 

will  be  seen  by  time  T is  1-e  and,  consequently,  the  average  amount 
X .j  is  actually  reduced  by  time  T is 

9 


\ 


i 


V 

-*! 


1 


V-  v— 


(19) 


-*lT 

d^Xfd  - e ) . 

From  (18)  we  see  that  at  time  T the  adjustment  procedure  decreases  Xi  by 
the  average  amount 

diXi.  (20) 

To  correct  for  this  bias  we,  therefore,  need  to  add  the  amount 

-X  i T 

d^x^e  (?1) 

to  the  adjustment  procedure  estimate  for  the  1-th  mode.  The  total  amount  of 
correction  needed  for  all  K modes  is  the  bias  term  B(T),  where  we  define 
B ( t ) , o < t < T,  by 


K -X<t 

B ( t)  = T Xjdje  . (22) 

1=1 

In  the  next  section,  we  will  develop  a methodology  for  estimating 
the  bias  term  B(T)  which  Is  appropriate  when  K,  the  number  of  problem  fail- 
ures modes.  Is  large.  This  estimate  of  the  bias  will  be  used  to  construct 
a+projection  procedure  which  is,  for  all  practical  purposes,  unbiased  when 
d^  equals  d^.  In  this  regard,  the  US  Anqy  Materiel  Systems  Analysis  Activity 
(USAMSAA)  has  conducted  studies  to  determine  from  historical  data  the  actual 
EFs  for  helicopters,  tanks,  missiles  and  electronic  equipment  (see  Trapnell, 
1982).  These  factors,  which  are  based  on  historical  experiences,  may  be 
used  as  guidelines  for  assigning  EFs  to  similar  systems  under  development. 

3.  A SYSTEM  RELIABILITY  PROJECTION  MODEL 

In  this  section,  we  present  a procedure  for  estimating  the  bias 
tern  B(T)  given  in  Equation  (22).  The  estimated  bias  will  then  be  used  to 
develop  a reliability  projection  methodology  which  Is  appropriate  under 
assumptions  which  are  generally  reasonable  in  practice  for  complex  system 
devel opnent. 

We  let  M(t)  denote  the  random  number  of  distinct  Type  B modes  ob- 
served during  (0,t)  and  observe  that  M(t)  may  be  written  as 

K 

M(t)  = l Ii(t).  (23) 


K -X,t 

E(M(t) ) = l (1  - e ).  (24) 

1=1 


We  denote  by  W(t)  the  cumulative  sum  of  the  EFs  for  the  Type  B 
modes  observed  during  (0,t).  It  follows,  of  course,  that 
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K 

W(t)  = l d-f 1 1 ( t ) (25) 

1 = 1 

and 

K -\ft 

E(W(t) ) = l di(1-e  ).  (26) 

i=l 

Taking  the  derivative  of  E(W(t))  yields 

K -X . t 

(d/dt)  E(U( t) ) = l djX.-e  1 . (27) 

1=1  1 1 

Thus,  the  key  result  that 

B(t)  = d/ot  E(W( t) ) . (28) 

To  develop  an  approach  for  estimating  B(t)  we  use  the  relation- 
ship (28)  and  develop  a model  which  expresses  E(W(t))  as  a power  function 
of  t,  namely. 


E(W( t) ) = atB  (29) 

where  a > 0,  6 > 0.  With  this  expression  for  E ( W( t) ) , we  have 

B( t)  = cStS-l . (30) 

To  formulate  the  model  we  begin  by  noting  that 

K - X i t 

h(t)  = d/dt  E(M(t))  = l Xie  (31) 

i =1 

is  the  avera;>.  rate  with  which  a new  Type  B mode  will  occur  at  time  t. 

We  will  motivate  the  non-homogeneous  Poisson  process  with  rate  h(t)  as  a 
stochastic  model  to  approximate  the  occurrence  of  new  Type  B modes.  This 
model  Implies  that  h( t) At  is  interpreted  as  the  unconditional  probability 
of  a new  Type  B mode  occurring  in  the  Interval  (t,t+At),  instead  of  an 
average  probability. 

Consider  an  interval  (t,t+At)  and  let  M(t,t+At)  * M(t+At)-M(t) 
denote  the  number  of  distinct  Type  B modes  observed  in  this  interval.  Note 
that  M( t,t+At)  Is  the  sum  of  K independent  Bernoulli  random  variables. 
Therefore,  for  E(M(t,t+At))  = E(M(t+At) )-E(M(t) ) small,  and  K large,  it  fol- 
lows (See  Feller,  1957,  Ch.  XI)  that  the  distribution  of  M(t,t+  At)  is 
approximately  Poisson  distributed. 

For  At  infinitesimally  small,  E(M(t,t+At))  is  approximately 
equal  to  h(t)At.  Thus,  for  At  infinitesimally  small,  the  Poisson  prob- 
ability of  at  least  one  Type  B mode  occurring  in  (t,t+At)  Is  approximately 
h(t)At,  and  the  Poisson  probability  of  more  than  one  distinct  Type  B mode 


"Vi 

S' 

■a 


11 


occurring  In  this  Interval  Is  near  zero.  In  addition,  for  large  K,  It  Is 
reasonable  to  assume  that  the  number  of  distinct  Type  B modes  observed  In 
nonoverlapping  Intervals  are  Independent.  Under  these  conditions,  It 
follows  that  the  occurrence  of  distinct  Type  B inodes  are  In  accordance 
with  the  nonhomogeneous  Poisson  process  with  mean  value  function  (24) 
and  Intensity  function  or  rate,  h(t). 

Based  on  empirical  studies  at  USAMSAA  on  complex  weapon  systems 
(see  Reiher,  et  al . , 1978,  page  49),  it  has  been  found  that  the  mean  number 
of  new  Type  B modes  by  time  t is  often  approximated  well  by  the  power  function 

E(M( t) ) * XtB.  (32) 

where  x > 0,  b > 0.  With  this  relationship,  we  have 

h(t)  = XBtS-1.  (33) 

Now,  W(t)  is  the  random  sum  of  the  EFs  for  those  Type  B modes 
observed  during  (0,t).  Within  a test  phase,  it  would  appear  reasonable  to 
assume  that  the  observed  EFs  may  be  treated  as  independent  random  variables, 
distributed  around  a common  mean,  say,  y<j,  and  independent  of  M(t).  Under 
these  assumptions,  we  have 


E(W(t) ) = v d E(M(t) ) 


(34) 


or 


E(Wit))  = yjXt®  * at®  (35) 

as  in  (29),  where  a = Xuj.  Consequently,  it  follows  that  the  bias  term 
B ( T ) is  expressed  as 


B(T)  = aeTe-1 . 

The  mean  uj  is  estimated  by 

1 M(T) 


= 


l d* 

M(T)  j=! 


(36) 


(37) 


where  the  d^.'s  are  the  EFs  for  the  M(T)  modes  observed.  To  estimate  the 

parameters  x and  8 for  the  rate  of  occurrence  of  new  Type  B modes,  one  may 
use  the  method  of  maximum  likelihood  (ML).  These  estimates  are  given  In 
Crow  (1974). 


Let  < X2  <...<  Xm  < T denote  the  cumulative  test  times  for  the 
first  occurrences  of  Type  B modes,  where  we  let  M ( T ) = M.  Then,  the  ML 
estimates  of  e and  x are 


M 

6 - m 


1 log(T/Xf) 


(38) 
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and 


(39) 


The  intensity  function  h { t > is  estimated  by 

h(t)  = ABt3"1  (40) 

for  t > 0.  In  particular  the  ML  estimate  for  the  rate  of  occurrence  for  dis- 
tinct Type  B modes  at  time  T is 

h(T)  = xbV'1  = . (41 ) 

T 


Further,  the  ML  estimate  of  the  bias  term  B(T)  is  given  by 


3(T) 


(42) 


Note  that  the  ML  estimate  of  a is 

a = Xpd. 


(43) 


Example 


In  this  example,  we  illustrate  the  ML  procedure  for  estimating 
the  occurrence  rate  h(t)  for  new  Type  B modes.  Suppose  a system  were 
tested  for  T = 400  hours  and  15  distinct  Type  B modes  were  first  observed 
at  the  following  cumulative  test  times:  0.2,  11.2,  37.2,  39.0,  48.4,  53.4, 
90.2,  91.6,  151.4,  159.4,  197.2,  240.2,  323.6,  361.2,  381.6.  These  data 
were  generated  by  computer  simulation  of  the  non-homogeneous  Poisson  pro- 
cess with  ,i ( c ) = Aet6-!  and  X = 0.42,  B = 0.5.  From  Equations  (39)  and 
(38),  the  ML  estimates  of  x and  g are  determined  to  be  A = 0.501, 

S - 0.567.  Using  these  estimates,  the  ML  estimate  of  h(t)  is  h(t)  = 
(0.50)(0.567)r0-433.  Evaluating  h(t)  at  t = 400  we  find  that  the  ML 
estimate  of  the  intensity  of  distinct  Type  B modes  at  the  end  of  test  is 
hi  400)  = 0.021. 

Crow  (1974)  shows  that  conditioned  on  M = m,  the  estimate 

HtlS  (44) 

M 

is  an  unbiased  estimate  of  6.  If  we  consider 
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m > 2 


Otherwi se. 


(45) 


as  an  estimate  of  h(T)  then,  for  Prob  (H  = 0 or  « = 1)  near  0,  h(T)  is  ap- 
proximately unbiased.  In  practice,  for  a complex  system  under  development, 
the  probability  of  two  or  more  problem  failure  modes  being  observed  Is  usu- 
ally near  unity.  In  this  case,  it  is  reasonable  to  use  h(T)  instead  of  h(T) 
for  estimating  the  rate  h(T). 


We  will  now  discuss  a procedure  for  determining  a projection  of 
r(T)  which  is  essentially  unbiased  under  the  assumptions  of  the  model  for 
estimating  B(T)  and  for  Prob  {M  = 0 or  M = 1 ( =0.  Let 


B(T)  = M<jh(T) 

and  consider  the  projection  7(T)  for  r(T)  where 

M 

r(T)  = 1/T  (NA  ♦ £ (1  - di > Ni)  + B(T). 


For  this  model,  we  have 

The  projected  mean  f me 
Example 


approximately  that 

E(r(T)  - r(T) ) = 0. 
between  failure  (MTBF)  is 
MTBF  = (r(T)H. 


(46) 


(47) 


(48) 

(49) 


We  illustrate  the  calculation  of  the  projection  r(T),  utilizing 
data  generated  by  computer  simulation  with  xA  = 0.02,  Xg  s 0.1,  K ■ 100 
and  the  d-j  ‘ s distributed  according  to  a Beta  distribution  with  mean  0.7. 

For  this  simulation  the  system  was  tested  for  T = 400  hours  and  experienced 
N = 42  failures.  Of  these  failures,  there  were  = 10  failures  which 
were  Type  A and  Ng  = 32  failures  which  were  Type  B.  In  addition,  the  32 
Type  B failures  were  due  to  M = 16  distinct  Type  B modes. 

The  cumulative  test  times  corresponding  to  the  occurrence  of  the 
Type  A modes  are:  43.16,  49.08,  75.62,  167.27,  238.73,  255.29,  277.33, 
350.28,  353.03,  367.68.  For  the  16  distinct  Type  B modes,  we  list  mode 
number  and  the  cumulative  failure  times  for  that  mode.  These  are: 

Mode  1 , 56.42,  72.09,  339.97 

Mode  2,  192.66 

Mode  3,  47.46,  350.2 
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Mode  4,  285.01 

Mode  5,  379.43 

Mode  6,  249.15,  324.47 

Mode  7,  133.43,  177.38,  324.95,  364.63 

Mode  8,  125.48,  164.66,  303.98 

Mode  9,  15.04,  254.99 

Mode  10,  111.99,  263.47,  373.03 

Mode  11,  53.96,  315.42 

Mode  12,  99.57,  274.71 

Mode  13,  25.26,  120.89,  366.27 

Mode  14,  388.97 

Mode  15,  395.25 

Mode  16,  100.31 


For  the  Type  B modes  listed  above,  the  assigned  EFs  are,  respec- 
tively, 0.87,  0.70,  0.77,  0.64,  0.72:  0.63,  0.74,  0.89,  0.67,  0.85,  0.77, 
0.92,  0.72,  0.69,  0.46,  0.50.  The  times  Xt  < X2  <...<  X15  of  first  occur- 
rence for  the  16  distinct  Type  B modes  are:  15.04,  25.26,  47.46,  53.96, 
56.42,  99.57.  100.31,  111.99,  125.48,  133.43,  192. Co,  249.15,  285.01, 
379.43,  388.97,  395.25. 

Using  Equations  ( 38 )_and  (37),  we  calculate  B 0.797  and  ud  = 
0.721.  The  unbiased  estimate  B of  B,  given  by  (44),  is  a = 0.747  and, 
from  (45),  the  estimated  rate  of  occurrence  of  new  Type  B modes  at  time  T 
is  TT( T ) = 0.030. 

To  calculate  the  projection  r'(T),  we  also  determine,  B(T)  = 0.022, 
using  (46),  N/\/T  = 0.025 


and 


M N, 

l d-d.-)  _L 
i =1  T 


0.020, 


where  Nj  is  the  number  of  observed  failures  for  the  i-th  mode  observed, 
i = 1 M . 

We  now  use  Equation  (47)  to  calculate  the  projected  estimate 
r(T).  This  gives 


7(T)  = 0.025  + 0.020  + 0.02?  * 0.067 


and  the  projected  system  MTBF  is 

MTBF  = (C.067)-1  = 14.9. 


(8),  is 


For  the  same  data,  the  adjustment  procedure  estimate  r*(T),  from 


r*(T)  = 0.025  + 0.020  = 0.045. 


The  correspond!' ng  MTBF  for  the  adjustment  procedure  is  [r*(T)]_^  = 22.2. 

This  is  an  estimate  of  the  upper  bound  on  the  achievable  MTBF  for  the  system 
and  is  not  an  estimate  of  the  current  system  MTBF.  The  actual  upper  bound, 
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using  ( E T r* ( T ) ] ) -1  given  by  Equation  (18)  and  the  parameters  of  the  simula- 
tion model,  is  20.6.  In  addition,  for  this  example,  it  can  be  stated  that 
1^,9  is  an  estimate  of  the  current  system  MTBF.  The  actual  MTBF  at  time  T, 

given  by  [r(T)]~^  from  Equation  (2),  is  14.7. 

4.  CONCLUSIONS 

In  this  report,  we  have  shown  that  the  adjustment  procedure  for 
reliability  projection  is  biased  and,  in  fact,  estimates  an  upper  bound  on 
system  reliability  after  all  problem  modes  have  been  found  and  fixed.  A 
model  was  developed  which  yields,  under  reasonable  assumptions,  an  unbiased 
estimate  of  the  true  system  reliability  after  delayed  fixes  have  been  in- 
corporated. It  is  emphasized  that  the  adjustment  procedure  should  not  be 
used  for  reliability  projections. 
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